Image processing device and image processing method

ABSTRACT

Distribution data is generated by a distribution data generation portion, a second range is specified by a specifying portion, the second range is mapped to a first range by a mapping portion, and third image data is generated by performing binarization based on a threshold value regulated in the first range by components, binarization can be suitably performed even in the case where distribution data of pixel values differs for each subject.

The present invention relates to an image processing apparatus forprocessing image data, for example, obtained by taking an image of asubject and an image processing method.

There is conventionally known an identifying device for performingindividual identifying processing, for example, by using image dataobtained by taking a picture of a living body (subject) (refer to, forexample, the Japanese Unexamined Patent Publication No. 10-127609).

In the above conventional identifying device, identifying processing isperformed, for example, by taking a picture of transmitted light of ahand of the subject and generating binarized image data based on apredetermined threshold value of pixel values of the image data. Forexample, the identifying device performs identifying processing based ona pattern indicating an arrangement of blood vessels in the binarizedimage data.

A distribution of pixel values of taken image data differs in eachsubject. For example, as to image data of a subject with much fatcomponent, the distribution data of pixel values spreads in a wide rangeand an average value of pixel values is relatively high comparing withimage data of a subject with less fat component.

Since the above conventional identifying device performs binarizationprocessing based on a predetermined threshold value, suitable binarizedimage data can be generated for image data of a subject with less fatcomponent, while there is a case where binarized data having lopsidedpixel values is undesirably generated for image data of a subject withmuch fat component and binarization processing cannot be performedsuitably, so that improvement is demanded.

Also, image data obtained by taking a picture of a subject includes verysmall regions equivalent to noise components, and the noise componentslargely affects on accuracy of identifying processing. Therefore, therehas been a demand to remove regions of a predetermined size equivalentto noise components from the image data.

Also, a linear pattern in the image data is significant in theidentifying processing, but the linear pattern is broken due to noise,etc. and cannot be visually recognized clearly in some cases. Therefore,there is a demand for obtaining image data including a clear linearpattern by connecting between pixel data close to each other to acertain extent by considering noise, etc.

SUMMARY OF THE INVENTION

An object of the present invention is to provide an image processingapparatus and an image processing method capable of suitably performingbinarization processing even in the case where distribution data ofpixel values differs in each subject.

Another object of the present invention is to provide an imageprocessing apparatus and an image processing method capable of removingregions being smaller than a predetermined size from image data obtainedby taking a picture of a subject and connecting pixel data close to acertain extent to each other.

To attain the above object, an image processing apparatus of a firstaspect of the present invention is an image processing apparatus,comprising: a distribution data generation means for generatingdistribution data indicating a distribution of pixel data for aplurality of pixel data indicating pre-regulated pixel values in thefirst range and composing first image data obtained by taking a pictureof a subject; a specifying means for specifying a second range to bebinarized in the first range based on the distribution data generated bythe distribution data generation means; a mapping means for mappingpixel data in the second range specified by the specifying means among aplurality of pixel data to the first range, and generating second imagedata composed of the mapped pixel data; and a binarization means ofbinarizing the second image data generated by the mapping means based ona threshold value regulated in the first range to generate a third imagedata.

According to the image processing apparatus of the first aspect of thepresent invention, the distribution data generation means generatesdistribution data indicating a distribution of pixel data for aplurality of pixel data indicating pre-regulated pixel values in thefirst range composing first image data obtained by taking a picture of asubject.

The specifying means specifies a second range to be binarized in thefirst range based on the distribution data generated by the distributiondata generation means.

The mapping means maps pixel data in the second range specified by thespecifying means among a plurality of pixel data to the first range, andgenerates second image data composed of the mapped pixel data.

The binarization means binarizes the second image data generated by themapping means based on a threshold value regulated in the first range togenerate third image data.

Furthermore, to attain the above objects, an image processing apparatusof a second aspect of the present invention is an image processingapparatus, comprising: a first processing means for indicating a pixelvalue and using as a pixel data the minimum pixel data in the firstregion around the pixel data, for each of a plurality of pixel datacomposing the first image data obtained by a taking a picture of asubject; and a second processing means for generating a second imagedata by using as the pixel data the maximum pixel data among pixel datain the second region larger than the first region around the pixel datafor each of image data by the first processing means.

Furthermore, to attain the above objects, an image processing method ofa third aspect of the present invention is an image processing method,including: a first step for generating distribution data indicating adistribution of pixel data for a plurality of pixel data indicatingpre-regulated pixel values in the first range and composing first imagedata obtained by taking a picture of a subject; a second step forspecifying a second range to be binarized in the first range based onthe distribution data generated by the first step; a third step formapping pixel data in the second range specified by the second stepamong a plurality of pixel data to the first range, and generates secondimage data composed of the mapped pixel data; and a fourth step forbinarizing the second image data generated by the third step on athreshold value regulated in the first range to generate third imagedata.

Furthermore, to attain the above objects, an image processing method ofa fourth aspect of the present invention is an image processing method,including: a first step for indicating a pixel value and using as apixel data the minimum pixel data in the first region around the pixeldata, for each of a plurality of pixel data composing the image dataobtained by a taking a picture of a subject; and a second step forgenerating a second image data by using as a pixel data the maximumpixel data among pixel data in the second region being larger than thefirst region around the pixel data by the first step.

According to the present invention, it is possible to provide an imageprocessing apparatus and an image processing method capable of suitablyperforming binarization processing even in the case where distributiondata of pixel values differs for each subject.

Also, according to the present invention, it is possible to provide animage processing apparatus and an image processing method capable ofremoving regions being smaller than a predetermined size from image dataobtained by taking a picture of a subject and connecting between pixeldata being close to a certain extent.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an overall schematic view of a first embodiment of a dataprocessing apparatus according to the present invention.

FIG. 2 is a block diagram in terms of hardware of the data processingapparatus shown in FIG. 1.

FIG. 3 is a block diagram of a function of the data processing apparatusshown in FIG. 1.

FIG. 4A to FIG. 4E are views for explaining an operation of the dataprocessing apparatus shown in FIG. 1. FIG. 4A is a view showing anexample of image data S11. FIG. 4B is a view showing an example of imagedata S1081. FIG. 4C is a view showing an example of distribution datad1. FIG. 4D is an enlarged view of distribution data. FIG. 4E is a viewshowing an example of image data S1084.

FIG. 5A and FIG. 5B are views for explaining an operation of a specificportion shown in FIG. 3. FIG. 5A is a view showing an example ofdistribution data d1. FIG. 5B is a view showing an example ofdistribution data d1′.

FIG. 6 is a flowchart for explaining an operation according to mappingprocessing of the data processing apparatus shown in FIG. 1.

FIG. 7 is a block diagram of a function according to filter processingof the data processing apparatus shown in FIG. 1.

FIG. 8 is a view for explaining a Gaussian filter.

FIG. 9A to FIG. 9F are views for explaining a Gaussian Laplacian filter.FIG. 9A is a view showing an example of step-shaped pixel values. FIG.9B is a view showing an example of pixel values. FIG. 9C is a viewshowing pixel values subjected to first-order differential processing.FIG. 9D is a view showing an example of pixel values. FIG. 9E is a viewshowing an example of pixel values subjected to primary differentialprocessing. FIG. 9F is a view showing an example of pixel valuessubjected to second-order differential processing.

FIG. 10A to FIG. 10C are views for explaining noise removing processingof the data processing apparatus shown in FIG. 1. FIG. 10A is a viewshowing an example of image data S1804. FIG. 10B is a view showing anexample of image data S 1805. FIG. 10C is a view showing an example ofimage data S 1806.

FIG. 11 is a flowchart for explaining an operation of the dataprocessing apparatus shown in FIG. 1.

FIG. 12A to FIG. 12D are schematic views for explaining an operation ofthe data processing apparatus shown in FIG. 1. FIG. 12A is a viewshowing an example of image data including noise components. FIG. 12B isa view showing an example of image data subjected to noise removingprocessing. FIG. 12C is a view showing an example of image data. FIG.12D is a view showing an example of image data subjected to connectionprocessing.

FIG. 13A to FIG. 13F are views for explaining degeneration processingand expansion processing of data processing apparatus shown in FIG. 1.FIG. 13A is a view showing an example of pixel data. FIG. 13B is anexample of pixel data when degeneration processing is performed based onpixels in a cross-shaped element. FIG. 13C is an example of pixel datawhen expansion processing is performed based on pixels in a cross-shapedelement. FIG. 13D is a view showing an example of pixel data. FIG. 13Eis an example of pixel data when degeneration processing is performedbased on pixels in 3×3 element.

FIG. 14A to FIG. 14C are views for explaining an operation of the dataprocessing apparatus shown in FIG. 1. FIG. 14A is a view showing anexample of image data S1807. FIG. 14B is a view showing an example ofimage data S1808. FIG. 14C is a view showing an example of image dataS1810.

FIG. 15 is a flowchart for explaining an operation of the dataprocessing apparatus shown in FIG. 1.

FIG. 16A to FIG. 16F are views for explaining an operation of a firstlow-pass filter processing of the data processing apparatus shown inFIG. 1. FIG. 16A is a view showing an example of a reference region in atwo-dimensional Fourier space. FIG. 16B is a view showing an example ofan enlarged reference region by predetermined magnification. FIG. 16C isa view showing an example of a low-pass filter. FIG. 16D is a viewshowing an example of image data. FIG. 16E is a view showing an exampleof image data subjected to low-pass filter processing. FIG. 16F is aview showing an example of image data subjected to binarizationprocessing.

FIG. 17A to FIG. 17E are views for explaining an operation of secondlow-pass filter processing of a low-pass filter portion. FIG. 17A is aview showing an example of a reference region in a two-dimensionalFourier space. FIG. 17B is a view showing an example of a low-passfilter. FIG. 17C is a view showing an example of image data. FIG. 17D isa view showing an example of image data subjected to low-pass filterprocessing. FIG. 17E is a view showing an example of image datasubjected to binarization processing.

FIG. 18A to FIG. 18E are views for explaining an operation of thirdlow-pass filter processing of the low-pass filter portion. FIG. 18A is aview showing an example of a reference region in the two-dimensionalFourier space. FIG. 18B is a view showing an example of a low-passfilter. FIG. 18C is a view showing an example of image data. FIG. 18D isa view showing an example of image data subjected to low-pass filterprocessing. FIG. 18E is a view showing an example of image datasubjected to binarization processing.

FIG. 19A to FIG. 19F are views for explaining an operation of thelow-pass filter portion of the data processing apparatus shown inFIG. 1. FIG. 19A is a view showing an example of image data S1810. FIG.19B is a view showing an example of image data S18102. FIG. 19C is aview showing an example of image data S18103. FIG. 19D is a view showingan example of image data S18102. FIG. 19E is a view showing an exampleof image data S18104. FIG. 19F is a view showing an example of imagedata S18105.

FIG. 20A to FIG. 20C are views for explaining an operation of thelow-pass filter portion of the data processing apparatus shown inFIG. 1. FIG. 20A is a view showing an example of image data S1804. FIG.20B is a view showing an example of image data S18106. FIG. 20C is aview showing an example of image data S1811.

FIG. 21 is a flowchart for explaining an operation of the low-passfilter portion of the data processing apparatus shown in FIG. 1.

FIG. 22A to FIG. 22C are views for explaining an operation of a maskportion and a skeleton portion of the data processing apparatus shown inFIG. 1. FIG. 22A is a view showing an example of a mask pattern. FIG.22B is a view showing an example of image data S1812. FIG. 22C is a viewshowing an example of image data S1813.

FIG. 23 is a flowchart for explaining an overall operation of the dataprocessing apparatus shown in FIG. 1.

FIG. 24 is a view for explaining a second embodiment of a remote-controldevice using a data processing apparatus according to the presentinvention.

FIG. 25 is a flowchart for explaining an operation of a remote-controldevice 1 a shown in FIG. 24.

FIG. 26 is a view for explaining a third embodiment of a data processingsystem using the data processing apparatus according to the presentinvention.

FIG. 27 is a flowchart for explaining an operation of the dataprocessing system shown in FIG. 26.

FIG. 28 is a view for explaining a fourth embodiment of a portablecommunication device using the data processing apparatus according tothe present invention.

FIG. 29 is a flowchart for explaining an operation of the dataprocessing apparatus shown in FIG. 28.

FIG. 30 is a view for explaining a fifth embodiment of the dataprocessing apparatus according to the present invention.

FIG. 31 is a flowchart for explaining an operation of a telephone shownin FIG. 30.

FIG. 32 is a view for explaining a sixth embodiment of the dataprocessing apparatus according to the present invention.

FIG. 33 is a view for explaining a seventh embodiment of the dataprocessing apparatus according to the present invention.

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS.

An image processing apparatus according to the present inventiongenerates distribution data indicating a distribution of pixel data,specifies a second range to be binarized, maps pixel data in the secondrange to a first range, generates image data composed of the mappedpixel data, and binarizes the image data based on a threshold valueregulated in the first range to generate binarized image data, based onimage data obtained by taking a picture of a subject, for a plurality ofpixel data composing the image data and indicating pixel values of afirst range regulated in advance.

Furthermore, an image processing apparatus according to the presentinvention eliminates regions being smaller than a predetermined sizefrom the image data obtained by taking a picture of the subject andconnects between pixel data close to a certain extent to each other.

Specifically, the image processing apparatus generates second image datafor each of the plurality of pixel data composing first image dataobtained by taking a picture of the subject and indicating pixel values,by using as the pixel data minimum pixel data among pixel data in thefirst region around the pixel data and, furthermore, using as pixel datamaximum pixel data among pixel data in a second region being larger thanthe first region around the pixel data for each of the pixel data.

Below, as a first embodiment of an image processing apparatus accordingto the present invention, a data processing apparatus for generatingimage data by taking a picture of a part with blood vessels of a livingbody of a subject h, extracting blood vessel information by performingimage processing on the image data, and performing authenticationprocessing based on the extracted blood vessel information will beexplained.

FIG. 1 is an overall schematic view of the first embodiment of the dataprocessing apparatus according to the present invention.

A data processing apparatus 1 according to the present embodimentcomprises, as shown in FIG. 1, an image pickup system 101, an extractionunit 102, and an authentication unit 103. The data processing apparatus1 corresponds to an example of an image processing apparatus accordingto the present invention.

The image pickup system 101 takes a picture of a subject h to generateimage data and outputs the image data as a signal S11 to the extractionunit 102.

The image pickup system 101 specifically comprises an irradiationportion 1011 and an optical lens 1012.

The irradiation portion 1011 is composed, for example, of a halogenlamp, etc. and irradiates an electromagnetic wave, for example a nearinfrared ray, to a part of the subject h by a control signal.

For example, when irradiating an electromagnetic wave to a living bodyas the subject h, a near infrared-ray from red to infrared ray having awavelength range of 600 nm to 1300 nm or so exhibits a high penetratingproperty comparing with that of electromagnetic waves of otherwavelength ranges. In this wavelength range, light absorption byhemoglobin in blood is dominant.

For example, when irradiating a near infrared ray from back of a hand asthe subject h and taking a picture of a transmitted light from the palmside, the electromagnetic wave is absorbed by hemoglobin in blood, sothat image data, wherein a region corresponding to thick blood vesselsnear the palm surface is darker than region other than the regioncorresponding to blood vessels, is obtained.

Vein of blood vessels is acquired in the process of growing up and theshape of the blood vessels largely varies between individuals. In thepresent embodiment, image data obtained by taking a picture of the bloodvessels is used as individually unique identification information inauthentication processing.

The optical lens 1012 focuses the transmitted light from the subject hon the image pickup unit 11.

The image pickup unit 11 generates image data S11 based on thetransmitted light focused by the optical lens 1012. For example, theimage pickup unit 11 is composed of a CCD (charge-coupled device) typeimage sensor and a C-MOS (complementary metal-oxide semiconductor) typeimage sensor, and outputs the image data S11 to the extraction unit 102.At this time, the image data S11 may be an RGB (red, green and blue)signal or image data of other colors than that and gray-scale, etc.

The extraction unit 102 performs image processing based on the imagedata S11, extracts image data used for authentication, such as skeletonimage data, and outputs this as a signal S102 to the authentication unit103.

The authentication unit 103 performs matching processing with registeredimage data stored in advance based on the signal S102 from theextraction unit 102 and performs authentication processing.

FIG. 2 is a block diagram in terms of hardware of the data processingapparatus shown in FIG. 1.

The data processing apparatus 1 comprises, for example as shown in FIG.2, an image pickup unit 11, an input unit 12, an output unit 13, acommunication interface (I/F) 14, a RAM (random access memory) 15, a ROM(read only memory) 16, a memory unit 17, and a CPU 18.

The image pickup unit 11, the input unit 12, the output unit 13, thecommunication interface (I/F) 14, the RAM 15, the ROM 16, the memoryunit 17 and the CPU (central processing unit) 18 are connected by a busBS.

The image pickup unit 11 is controlled by the CPU 18 and generates imagedata of the subject h and outputs this as a signal S11.

The input unit 12 outputs a signal, for example, in accordance with anoperation of a user to the CPU 18. For example, the input unit 12 iscomposed of a keyboard, a mouse and a touch panel, etc.

The output unit 13 is controlled by the PCU 18 and performs outputtingin accordance with predetermined data. For example, the output unit 13is composed of a display or other display devices.

The communication interface (I/F) 14 is controlled by the CPU 18 andperforms data communication with other data processing apparatuses, forexample, via a not shown communication network.

The RAM 15 is for example used as a work space of the CPU 18.

The ROM 16 stores data, such as initial values and initial parameters,and the data is used by the CPU 18.

In the memory unit 17, predetermined data is written and read by the CPU18. For example, the memory unit 17 is composed of an HDD (hard diskdrive) and other memory devices.

The memory unit 17 comprises, for example as shown in FIG. 2, a programPRG and image data D_P, etc.

The program PRG includes functions according to an embodiment of thepresent invention, such as functions of the extraction unit 102 and theauthentication unit 103, and the functions are realized by beingexecuted by the CPU 18.

The image data D_P is image data, such as registered image data, forexample, used in the authentication processing.

FIG. 3 is a block diagram of functions of the data processing apparatusshown in FIG. 1.

For example, the CPU 18 realizes as functions of the extraction unit102, functions of a gray-scale conversion portion 1801, a distributiondata generation portion 1802, a specifying portion 1803, a mappingportion 1804, a Gaussian filter 1805, a Gaussian Laplacian 1806, a firstdegeneration processing portion 1807, a first expansion processingportion 1808, a second expansion processing portion 1809, a seconddegeneration processing portion 18010, a low-pass filter portion 1811, amask portion 1812 and a skeleton portion 1813, by executing the programPRG as shown in FIG. 3.

The present invention is not limited to this embodiment. For example,the functions of components shown in FIG. 3 may be realized by hardware.

The distribution data generation portion 1802 corresponds to an exampleof a distribution data generation means according to the presentinvention, and the specifying portion 1803 corresponds to an example ofa specifying means according to the present invention.

The mapping portion 1804 corresponds to an example of a mapping meansaccording to the present invention, and the low-pass filter portion 1811corresponds to an example of a filter processing means according to thepresent invention.

The Gaussian filter 1805, the Gaussian Laplacian 1806, the firstdegeneration processing portion 1807, the first expansion processingportion 1808, the second expansion processing portion 1809, the seconddegeneration processing portion 1810, the low-pass filter portion 1811,the mask portion 1812 and the skeleton portion 1813 correspond to anexample of a binarization means according to the present invention.

The first degeneration processing portion 1807 corresponds to a firstprocessing means according to the present invention, the first expansionprocessing portion 1808 corresponds to an example of a fourth processingmeans according to the present invention, the second expansionprocessing portion 1809 corresponds to an example of a second processingmeans according to the present invention, and the second degenerationprocessing portion 1810 corresponds to an example of a third processingmeans according to the present invention.

The gray-scale conversion portion 1801 converts the RGB signal S11 fromthe image pickup unit 11 to be gray-scale and outputs as a signal S1801to the distribution data generation portion 1802. Specifically, thegray-scale conversion portion 1801 converts the RGB signal topredetermined tones from white to black, for example, 256 tones.

In the present embodiment, the image pickup unit 11 generates the RGBsignal S11 and the gray-scale conversion portion 1801 performsconversion processing to gray scale on the signal S11, but the presentinvention is not limited to this embodiment. For example, when the imagepickup unit 11 generates gray scale image data S11, the gray scaleconversion portion 1801 is not provided.

FIG. 4A to FIG. 4E are views for explaining an operation of the dataprocessing apparatus shown in FIG. 1.

In the present embodiment, the image pickup unit 11 takes a picture of,for example, a finger of a living body of the subject h and outputs anRGB image data S11 as shown in FIG. 4A.

The gray scale conversion portion 1801 generates gray scale image dataS1802, for example, as shown in FIG. 4B based on the image data S11 andoutputs to the distribution data generation portion 1802.

The distribution data generation portion 1802 generates distributiondata d1 indicating a distribution of pixel data based on the signalS1801 from the gray scale conversion portion 1801, for a plurality ofpixel data composing image data and indicating pixel values in the firstrange regulated in advance, and outputs as a signal S1802 to thespecifying portion 1803.

Specifically, when assuming the abscissa “c” is a value of tones (alsoreferred to as a pixel value) and the ordinate “f” is the number of thepixel data (also referred to as a degree), the distribution datageneration portion 1802 generates histogram as distribution data d1 forpixel data indicating pixel values of a range of 256 tones as a firstrange r1 as shown in FIG. 4C based on the signal S1801. In FIG. 4C, asmall pixel value corresponds to black, and a large pixel valuecorresponds to white.

Specifically, the distribution data generation portion 1802 generatesdistribution data d1 indicating the number of pixel data having pixelvalues for the respective pixel values in the first range r1.

FIG. 5A and FIG. 5B are views for explaining an operation of thespecifying portion shown in FIG. 3.

The specifying portion 1803 specifies, based on the signal S1802, arange with the maximum pixel value or less among the pixel values ofpredetermined number of pixel data in the first range r1 as a secondrange r2 to be binarized, and outputs this as a signal S1803.

Specifically, for example as shown in FIG. 5A, the specifying portion1803 specifies as a second range r2 a range with the maximum pixel ofvalue r11 or less among pixel values r11, r12, r13 and r14 by the numberof predetermined threshold value V_th in the first range r1.

For example, the specifying portion 1803 specifies as a second range r2a range with pixel values of 1 to 110 in the case of the distributiondata d1 as shown in FIG. 5A.

Distribution data of pixel values of the subject h differs in eachsubject h. For example, comparing with image data of a subject with lessfat component, the histogram d1′ of image data of a subject with muchfat component exhibits distribution data d1′ spreading in a wide rangeand has a relatively high average value of pixel values as shown in FIG.5B.

For example, in the case of the distribution data d1′ as shown in FIG.5B, the specifying portion 1803 specifies as a second range r2′ a rangewith pixel values of not more than the maximum pixel value r11′ amongthe pixel values r11′, r12′, r13′ and r14′ by the number of apredetermined threshold value V_th in the first range r1.

The mapping portion 1804 maps pixel data in the second range r2specified by the specifying portion 1803 among the plurality of pixeldata to the first range r1, generates second image data composed of themapped pixel data, and outputs this as a signal S1804.

Specifically, for example, when assuming that a range of pixel values of0 to 110 is a second range r2 as shown in FIG. 4C, the mapping portion1804 performs mapping by enlarging the pixel data to the first range r1as a range of pixel values of 0 to 256 as shown in FIG. 4D, enlarges thecenter portion in image data not including blood vessel information asshown in FIG. 4E, and generates second image data S1804.

FIG. 6 is a flowchart for explaining an operation according to themapping processing of the data processing apparatus shown in FIG. 1.With reference to FIG. 4, FIG. 5A, FIG. 5B and FIG. 6, an operation ofthe distribution data generation portion 1802, the specifying portion1803 and the mapping portion 1804 will be explained.

The image pickup unit 11 takes a picture of the subject h and outputsimage data S11 to the gray scale conversion portion 1801. The image dataS11 is converted to gray scale of 256 tones by the gray scale conversionportion 1801 and input as a signal S1801 to the distribution datageneration portion 1802.

In a step ST1, for example as shown in FIG. 4C, based on the signalS1801, for a plurality of pixel data composing image data S andindicating pixel values in the first range r1 regulated in advance, thedistribution data generation portion 1802 generates distribution data d1indicating the number of pixel data having the pixel values and outputsas a signal S1802 to the specifying portion 1803.

In a step ST2, as shown in FIG. 4C, based on the signal S1802, thespecifying portion 1803 specifies as a second range r2 to be binarized arange with pixel values of not more than the maximum pixel value r11among pixel values in pixel data by the predetermined number, forexample a threshold value V_th, in the first range r1, and outputs as asignal S1803 to the mapping portion 1804.

In a step ST3, as shown in FIG. 4D, the mapping portion 1804 maps pixeldata in the second range r2 specified by the specifying portion 1803among the plurality of pixel data to the first range r1 based on thesignal S1803, generates second image data composed of the mapped pixeldata and outputs as a signal S1804.

In a step ST4, the second image data S1804 generated in the mappingportion 1804, for example, by later explained components 1085 to 1812,etc. is binarized based on the regulated threshold value regulated inthe first range r1, for example 100 tones, and generates third imagedata.

As explained above, in the present embodiment, for example as shown inFIG. 4C and FIG. 4D, as a result that distribution data is generated bythe distribution data generation portion 1802, a second range isspecified by the specifying portion 1803, the pixel data in the secondrange is mapped to the first range by the mapping portion 1804, andimage data is generated by binarization based on a threshold valueregulated in the first range r1 by the later explained components 1805to 1812; binarization processing can be suitably performed even whendistribution data d1 differs in each subject h.

Also, since the pixel data in the specified second range is mapped tothe first range, the contrast becomes high and binarization processingcan be suitably performed.

The data processing apparatus 1 according to the present embodimentperforms edge enhancement processing after performing noise removingprocessing on the image data generated in the above steps. For example,the data processing apparatus 1 performs any processing among aplurality of different noise removing processing based on the signalS1804, and performs edge enhancement processing after the noise removingprocessing.

FIG. 7 is a block diagram of a function according to filter processingof the data processing apparatus shown in FIG. 1.

The CPU 18 realizes functions of a selection portion 1814 and aplurality of noise removing filters 1815 shown in FIG. 7, for example,by executing the program PRG.

The noise removing filter 1815 corresponds to an example of a noiseremoving means according to the present invention.

The selection portion 1814 outputs to the noise removing filter 1815 asignal S1814 for selecting any noise removing filter among a pluralityof noise removing filters for performing different noise removingprocessing among the noise removing filter 1815.

For example, the selection portion 1814 detects noise distributioncharacteristics of the signal S1804 and outputs a signal S1814 forselecting a noise removing filter suitable to the noise characteristicsbased on the detection result.

Also, for example, the selection portion 1814 may output a signal S1814for selecting the noise removing filter based on the signal from theinput unit 12 in accordance with an operation of a user.

The noise removing filter 1815 comprises a plurality of filters fornoise removing processing, for example, a Gaussian filter 1815_1, amedian filter 1815_2, a maximum value filter 1815_3, a minimum valuefilter 1815_4, a two-dimensional adaptive noise removing filter 1815_5,a proximity filter 1815_6, an averaging filter 1815_7, a Gaussianlow-pass filter 1815_8, a two-dimensional Laplacian proximity filter1815_9, and a Gaussian Laplacian filter 1815_10; selects any (at leastone) noise removing filter, for example, in accordance with the signalS1814 from the selection portion 1814, performs noise removingprocessing on the signal S1804 with the selected noise removing filter,and generates image data S1806.

Below, the filter processing will be explained. Generally, filterprocessing is performed with a filter h (n1, n2) on image data u (n1,n2), wherein a grid point (n1, n2) on the two-dimensional plane is avariable, and image data v (n1, n2) is generated as show in the formula(1). Here, convolution integral is indicated as “*”.

$\begin{matrix}\begin{matrix}{{v\left( {n_{1},n_{2}} \right)} = {{u\left( {n_{1},n_{2}} \right)}*{h\left( {n_{1},n_{2}} \right)}}} \\{= {\sum\limits_{m\; 1}\;{\sum\limits_{m\; 2}\;{{h\left( {m_{1},m_{2}} \right)}{u\left( {{n_{1} - m_{1}},{n_{2} - m_{2}}} \right)}}}}} \\{= {\sum\limits_{m\; 1}\;{\sum\limits_{m\; 2}\;{{u\left( {m_{1},m_{2}} \right)}{h\left( {{n_{1} - m_{1}},{n_{2} - m_{2}}} \right)}}}}}\end{matrix} & (1)\end{matrix}$

The Gaussian filter 1815_1 performs convolution processing on a Gaussfunction hg (n1, n2) as shown in the formula (2), for example, by usinga standard deviation σ. Specifically, as shown in the formulas (3) and(1), noise removing processing is performed by using the Gaussian filterh (n1, n2).h _(g)(n _(1,) n ₂)=e ^(−(n) ¹ ² ⁺ ² ² ^()/(2σ) ²⁾   (2)

$\begin{matrix}{{h\left( {n_{1},n_{2}} \right)} = \frac{h_{g}\left( {n_{1},n_{2}} \right)}{\;{\sum\limits_{n\; 1}\;{\sum\limits_{n\; 2}\; h_{g}}}}} & (3)\end{matrix}$

FIG. 8 is a view for explaining a Gaussian filter.

The Gaussian filter 1815_1 is a smoothing filter and performs smoothingprocessing by calculating by weighting in accordance with a twodimensional Gauss distribution, wherein focused pixel data is at thecenter, for example, as shown in FIG. 8. For example, focused pixel datais (0, 0) in FIG. 8.

For example, when arranging pixel data in a local region of n×n, whereinthe focused pixel data is at the center, the median filter 1815_2 uses apixel value of the pixel data in the middle of the order as a pixelvalue of the focused pixel data.

The maximum value filter 1815_3 uses a pixel value of the maximum valueas a pixel value of the focused pixel data, for example, among pixeldata of a local region of n×n, wherein the focused pixel is at thecenter.

The minimum value filter 1815_4 uses a pixel value of the minimum valueas a pixel value of the focused pixel data, for example, among pixeldata of a local region of n×n, wherein the focused pixel is at thecenter.

The two-dimensional adaptive noise removing filter 1815_5 is, forexample, a so-called Wiener filter and performs filter processing tominimize a mean square error with respect to image data based on theimage data to improve the image.

The proximity filter 1815_6 is filter processing for calculating anoutput pixel based on a pixel value of, for example, n×n pixel amongimage data. Specifically, for example, the proximity filter 1815_6performs filter processing based on the maximum value, minimum value andstandard deviation from a proximity value in accordance with the data.

The averaging filter 1815_7 performs filter processing by calculating anaverage value of pixel values of, for example, n×n pixel among the imagedata and using the same as an output pixel.

The Gaussian low-pass filter 1815_8 performs noise removing andsmoothing processing. Specifically, the Gaussian low-pass filter 1815_8performs smoothing processing on image data based on Gaussian typeweighting.

The two-dimensional Laplacian proximity filter 1815_9 performssecond-order differential processing to perform edge detection based onthe image data.

The Gaussian Laplacian filter 1815_10 performs filter processing whereina Gaussian filter calculates a Laplacian (second-order differential). Adetailed explanation will be given below.

The Laplacian can be expressed, for example, as shown in the formula (4)in the two-dimension Euclidean coordinate system.

$\begin{matrix}{\nabla^{2}{= {\frac{\partial^{2}}{\partial x^{2}} + \frac{\partial^{2}}{\partial y^{2}}}}} & (4)\end{matrix}$

Also, the Laplacian can be expressed in matrix of 3×3 as shown in theformula (5), for example, by using a predetermined value α. Here, thefocused pixel is made to be the center of the matrix.

$\begin{matrix}{\nabla^{2}{= {\frac{4}{\left( {a + 1} \right)}\begin{bmatrix}\frac{\alpha}{4} & \frac{1 - \alpha}{4} & \frac{\alpha}{4} \\\frac{1 - a}{4} & {- 1} & \frac{1 - \alpha}{4} \\\frac{\alpha}{4} & \frac{1 - \alpha}{4} & \frac{\alpha}{4}\end{bmatrix}}}} & (5)\end{matrix}$

The Laplacian of a Gaussian filter performs convolution processing onthe Gauss function hg (n1, n2) as shown in the formula (6), for example,by using the standard deviation σ. Specifically, as shown in theformulas (7) and (1), noise removing processing is performed by usingthe Gaussian Laplacian filter h (n1, n2).h _(g)(n _(1,) n ₂)=e ^(−(n) ¹ ² ^(+n) ² ² ^()/(2σ) ² ⁾  (6)

$\begin{matrix}{{h\left( {n_{1},n_{2}} \right)} = \frac{\left( {n_{1}^{2} + n_{2}^{2} - {2\sigma^{2}}} \right){h_{g}\left( {n_{1},n_{2}} \right)}}{2{\pi\sigma}^{6}{\sum\limits_{n\; 1}\;{\sum\limits_{n\; 2}\; h_{g}}}}} & (7)\end{matrix}$

Also, the Laplacian of the Gaussian filter can be expressed, forexample, as shown in the formula (8) when expressed in matrix by using apredetermined value α. Here, the focused pixel is made to be at thecenter of the matrix.

$\begin{matrix}{\frac{1}{\left( {a + 1} \right)}\begin{bmatrix}{- \alpha} & {\alpha - 1} & {- \alpha} \\{\alpha - 1} & {\alpha + 5} & {\alpha - 1} \\{- \alpha} & {\alpha - 1} & {- \alpha}\end{bmatrix}} & (8)\end{matrix}$

FIG. 9A to FIG. 9F are views for explaining a Gaussian Laplacian filter.For a plane explanation, the image data is assumed to beone-dimensional.

An edge is a boundary of a plane and a plane generated by a change ofpixel values (brightness). An edge can be detected by performing spatialdifferential. For example, there are first-order differential andsecond-order differential.

For example, the case of a step-shaped pixel value f(x) shown in FIG. 9Awill be explained. Here, the ordinate axis is a pixel value and theabscissa axis is the x-axis.

Specifically, an edge region continuously changes with a predeterminedwidth L between the first pixel value f1 and the second pixel value f2as shown in FIG. 9B. When assuming that the image data f(x) isfirst-order differential processing, it sharply changes at apredetermined width L in the boundary region, for example, as shown inFIG. 9C.

For example, edge detection processing detects an abrupt change of imagef′(x) after the first-order differential processing and specifies theedge.

Also, the edge detection processing may perform detection by quadraticdifferential processing (Laplacian).

For example, in the case where image data is a pixel value f(x) shown inFIG. 9D, a first-order differential value f′(x) shown in FIG. 9E and asecond-order differential value f″(x) shown in FIG. 9F shown in FIG. 9Fare obtained.

The second-order differential value f″(x) changes its sign at a pointwhere tilt is the largest on a slope of the edge. Accordingly, a pointwhere the second-order differential crosses with the x-axis (referred toas a zero cross point) P_cr indicates the edge position. The image datais two-dimensional data and specifies as an edge a position of the zerocrossing point P_cr among image data subjected to second-orderdifferential processing at the time of actual edge detection.

For example, the case where the selection portion 1814 selects theGaussian filter 1815_1 and the Gaussian Laplacian filter 1815_10 fornoise removing processing will be explained. For example as shown inFIG. 3, it is assumed that a Gaussian filter 1805 is the Gaussian filter1815_1 and a Gaussian Laplacian filter 1806 is the Gaussian Laplacianfilter 1815_10.

FIG. 10A to FIG. 10C are views for explaining noise removing processingof the data processing apparatus shown in FIG. 1. FIG. 11 is a flowchartfor explaining an operation of the data processing apparatus shown inFIG. 1. With reference to FIG. 10A to FIG. 10C and FIG. 11, an operationof the data processing apparatus, particularly an operation regardingnoise removing processing will be explained.

In a step ST11, for example, the selection portion 1814 detects noisedistribution characteristics of a signal S1804 and outputs a signalS1814 to select a noise removing filter suitable to the noisecharacteristics based on the detected results to the noise removingfilter 1815. For example, the selection portion 1814 outputs to thenoise removing filter 1815 a signal S1814 to select the Gaussian filter1815_1 and the Gaussian Laplacian filter 1815_10 for the noise removingprocessing.

In a step ST12, any (at least one) noise removing filter is selectedbased on the signal S1814 in the noise removing filter 1815, noiseremoving processing is performed on the signal S1814 by the selectednoise removing filter, and image data S1806 is generated.

For example, the noise removing filter 1815 selects the Gaussian filter1815_1 and the Gaussian Laplacian filter 1815_10. For convenience of theexplanation, the Gaussian filter 1815_1 and the Gaussian Laplacianfilter 1815_10 are respectively explained as the Gaussian filter 1805and the Gaussian Laplacian filter 1806.

In the step ST12, the Gaussian filter 1805 performs noise removingprocessing shown in the formulas (1) and (3), for example, based on thesignal S1804 shown in FIG. 10A, generates image data S1805, for example,shown in FIG. 10B, and outputs to the Gaussian Laplacian filter 1806.

In a step ST13, the Gaussian Laplacian filter 1806 performs edgeenhancement processing based on the signal S1805, for example, as shownin FIG. 10B, generates image data S1806, for example, as shown in FIG.10C, and outputs the same. The image data S1806 is binarized image data.

The Gaussian Laplacian filter 1806 performs binarization processingbased on a threshold value regulated in the first range r1, for exampleshown in FIG. 4C, when performing binarization processing.

As explained above, as a result that the selection portion 1814 forselecting any noise removing processing among a plurality of noiseremoving processing and as noise removing filters 1815, for example, theGaussian filter 1815_1, the median filter 1815_2, the maximum valuefilter 1815_3, the minimum value filter 1815_4, the two-dimensionaladaptive noise removing filter 1815_5, the proximity filter 1815_6, anaveraging filter 1815_7, the Gaussian low-pass filter 1815_8, thetwo-dimensional Laplacian proximity filter 1815_9, and the GaussianLaplacian filter 1815_10 are provided; and, for example, a filterselected by the selection portion 1814 performs the noise removingprocessing based on the signal S1804; then, edge enhancement processingis performed by the Gaussian Laplacian filter 1806 for binarization; itis possible to generate suitably binarized image data based on thepredetermined threshold value of the first range r1 by removing, forexample, noises caused by diffused reflection of a body of a subject h,the image pickup unit 11 and other devices, from the image data S1804.

Also, since the selection portion 1814 selects a filter in accordancewith the noise characteristics, noises can be removed with highaccuracy.

Also, for example, by performing Gaussian filter processing and GaussianLaplacian filter processing on the image data generated by taking apicture of light transmitted through a part including blood vessels ofthe subject h, noises can be removed with high accuracy, binarizationprocessing can be suitably performed and it is possible to generate animage wherein a pattern indicating blood vessels can be visuallyrecognized.

FIG. 12A to FIG. 12D are schematic views for explaining an operation ofthe data processing apparatus shown in FIG. 1.

The data processing apparatus 1 according to the present embodimentperforms removing processing as shown in FIG. 12B on a pixel of a noisecomponent which is smaller than a predetermined sized region ar_th1, forexample, as shown in FIG. 12A based on the binarized image data S1806generated in the above processing.

Also, the data processing apparatus 1 performs processing of connectingpixel data g21 and g22 having the same pixel value within apredetermined distance ar_th2, for example, based on the binarized imagedata S1806 shown in FIG. 12C, and generates image data having a linearpattern g2, for example, shown in FIG. 12D. In the present embodiment,the linear patter corresponds to an example of a pattern indicatingblood vessels.

Specifically, the data processing apparatus 1 performs degenerationprocessing by using the minimum pixel data among pixel data in the firstregion around the pixel data as predetermined pixel data for each of aplurality of pixel data composing image data and indicating pixelvalues, and expansion processing by using the maximum pixel data amongpixel data in the second region being larger than the first regionaround the pixel data as predetermined pixel data for each pixel data bythe degeneration processing; and generates image data including a linearpattern.

In the present embodiment, the above functions are realized, forexample, by using the Morphology function.

FIG. 13A to FIG. 13F are views for explaining the degenerationprocessing and expansion processing of the data processing apparatusshown in FIG. 1.

Based on the image data S1806, for each of the plurality of pixel datacomposing the image data S1806 and indicating pixel values, a firstdegeneration (erode) processing portion 1807 generates image data S1807by using the minimum pixel data among pixel data in the first regionaround the pixel data as predetermined pixel data, and output this tothe first expansion processing portion 1808.

Specifically, for example as shown in FIG. 13A, the first degenerationprocessing portion 1807 uses the minimum pixel data among pixel data ina cross-shaped element EL1 as a first region, wherein focused pixel datag_att is at the center, as a pixel value of the focused pixel g_att. Inthe present embodiment, as shown in FIG. 13B, the minimum value “0” isused as the focused pixel data g_att.

Based on the image data S1807, for each of a plurality of pixel datacomposing the image data S1807 and indicating pixel values, the firstexpansion (dilate) processing portion 1808 generates image data S1808 byusing the maximum pixel data among pixel data in the first region aroundthe pixel data as predetermined pixel data, and outputs the same to thesecond expansion processing portion 1809.

Specifically, for example as shown in FIG. 13A, the first expansionprocessing portion 1808 uses the maximum pixel data among pixel data inthe cross-shaped element EL1 as a first region, wherein the focusedpixel data g_att is at the center, as a pixel value of the focused pixelg_att. In the present embodiment, as shown in FIG. 13C, the maximumvalue 1 is used as the focused pixel data g_att.

Based on the image data S1808, for each of a plurality of pixel datacomposing the image data S1808 and indicating a pixel value, the secondexpansion processing portion 1809 generates image data S1809 by using aspredetermined pixel data the maximum pixel data among pixel data in thesecond region being larger than the first region around the pixel data,and outputs to the second degeneration processing portion 1810.

Specifically, the second expansion processing portion 1809 uses as apixel value of the focused pixel g_att the maximum pixel data amongpixel data in a 3×3 rectangular shaped element EL2, wherein the focusedpixel data g_att is at the center, as a second region being larger thanthe first region, for example as shown in FIG. 13D. In the presentembodiment, for example as shown in FIG. 13E, the maximum value 1 isused as the focused pixel data g_att.

In the present embodiment, an explanation will be made by taking a 3×3element as an example, but the present invention is not limited to theembodiment. For example, it may be a desired size of 5×5 and 7×7, etc.

Based on the image data S1809, for each of a plurality of pixel datacomposing the image data S1809 and indicating pixel values, the seconddegeneration processing portion 1810 generates image data S1810 by usingas predetermined pixel data the minimum pixel data among pixel data inthe second region being larger than the first region around the pixeldata.

Specifically, the second degeneration processing portion 1810, forexample as shown in FIG. 13D, the minimum pixel data among pixel data inthe 3×3 rectangular element EL2, wherein the focused pixel data g_att isat the center, as the second region being larger than the first region,is used as a pixel value of the focused pixel g_att. In the presentembodiment, as shown in FIG. 13F, the minimum value 0 is used as thefocused pixel data g_att.

FIG. 14A to FIG. 14C are views for explaining an operation of the dataprocessing apparatus shown in FIG. 1. FIG. 15 is a flowchart forexplaining an operation of the data processing apparatus shown inFIG. 1. With reference to FIG. 10C, FIG. 14A to FIG. 14C and FIG. 15, anoperation of the data processing apparatus will be explained byparticularly focusing on degeneration processing and expansionprocessing.

In a step ST21, based on the image data S1806, for example shown in FIG.10C, the first degeneration processing portion 1807 generates imageS1807 as shown in FIG. 14A by using as a pixel value of the focusedpixel g_att the minimum pixel data among pixel data in a cross-shapedelement EL1 as the first region, wherein the focused data is at thecenter, for example as shown in FIG. 13A.

As a result of the first degeneration processing, the first degenerationprocessing portion 1807 generates image data S1807, wherein pixel databeing smaller than a predetermined size is removed.

In a step ST22, based on the image data S1807, for example shown in FIG.14A, the first expansion processing portion 1808 generates image dataS1808 shown in FIG. 14B by using as a pixel value of the focused pixelg_att the maximum pixel data among pixel data in the cross-shapedelement EL1 as the first region, wherein the focused pixel data g_att isat the center, for example as shown in FIG. 13A.

In a step ST23, based on image data S1808, for example shown in FIG.14B, the second expansion processing portion 1809 generates image dataS1808 by using as a pixel value of the focused pixel g_att the maximumpixel data among pixel data in a 3×3 rectangular shaped element EL2,wherein the focused pixel data g_att is at the center, as a secondregion being larger than the first region, for example as shown in FIG.13D.

From the processing in the above steps ST22 and ST23, the firstexpansion processing portion 1808 and the second expansion processingportion connect pixel data having the same pixel value within apredetermined distance ar_th2 and generates image data having a linearpattern.

In a step ST24, for example based on image data S1809, the seconddegeneration processing portion 1810 generates image data S1810, forexample, as shown in FIG. 14C by using as a pixel value of the focusedpixel g_att the minimum pixel data among pixel data in a 3×3 rectangularshaped element EL2, wherein the focused pixel data g_att is at thecenter, as a second region being larger than the first region, forexample as shown in FIG. 13D.

As explained above, for each of a plurality pixel data composing theimage data S1806 and indicating pixel values, the first degenerationprocessing portion 1807 for generating image data S1807 by using aspredetermined pixel data the minimum pixel data among pixel data in thefirst region around the pixel data, the first expansion processingportion 1808 for generating image data S1808 by using as predeterminedpixel data the maximum pixel data among pixel data in the first regionaround the pixel data, the second expansion processing portion 1809 forgenerating image data S1809 by using as predetermined pixel data themaximum pixel data among pixel data in the second region being largerthan the first region around the pixel data, and the second degenerationprocessing portion 1810 for generating image data S1810 by using aspredetermined pixel data the minimum pixel data among pixel data in thesecond region being larger than the first region around the pixel data;it is possible to leave a linear pattern and fine pattern can be removedas noise components.

The low-pass filter portion 1811 performs filter processing for leavinga linear pattern, for example, based on the image data S1810 andgenerates image data S1811.

Specifically, the low-pass filter portion 1811 specifies low frequencycomponent data than a threshold value for leaving the linear pattern byfrequency components in the two-dimensional Fourier space obtained byperforming two-dimensional Fourier transform processing on the imagedata S1810, performs inverse two-dimensional Fourier transformprocessing on the specified data, and generates image data S1811.

FIG. 16A to FIG. 16F are views for explaining an operation of firstlow-pass filter processing of the data processing apparatus shown inFIG. 1. With reference to FIG. 16A to FIG. 16F, an operation of thelow-pass filter portion 1811 will be explained.

The low-pass filter portion 1811 according to the present embodimentperforms low-pass filter processing by changing a threshold value for aplurality of times, for example three times, for highly accuratelyextracting a linear pattern.

A threshold value of frequency components for leaving the linear patternwill be explained.

For example, when assuming that the abscissa axis is x components in theFourier space and the ordinate axis is y components in the Fourierspace, the low-pass filter portion 1811 sets a region ar_ref to be areference of a threshold value in the Fourier space as shown in FIG.16A.

In the present embodiment, for example as shown in FIG. 16A, a rhombicreference region ar_ref is set in the 360×360 Fourier space, wherein theorigin point 0 is set to be the center. As shown in FIG. 16B, a regionar_ref′ including the reference region ar_ref and obtained by enlargingthe reference region by a predetermined magnification is set, and theregion ar_ref′ is used as a low-pass filter.

In the first low-pass filter processing, for example as shown in FIG.16C, a low-pass filter ar_LPF1 is set, so that a region ar_h indicatinghigh frequency components is cut in the Fourier space. The region ar_hcorresponds, for example, to a geometrically symmetric pattern and anapproximate circular pattern, etc. in an real-space. By cutting theregion ar_h, the above geometrically symmetric pattern can be removed.

As the threshold value, for example as shown in FIG. 16C, a regionar_LPF1 surrounded by (180, 150), (150, 180), (−150, 180), (−180, 150),(−180, −150), (−150, 180), (150, −180) and (180, −150) in thetwo-dimensional Fourier space is set. The region ar_LPF1 corresponds,for example, to a linear pattern in an actual space. By specifying theregion ar_LPF1, the linear pattern can be specified.

The low-pass filter portion 1811 specifies low frequency component datain the region ar_LPF1 in the Fourier space as shown in FIG. 16C based onimage data S101, for example shown in FIG. 16D, as the image data. Forexample, when the specified low frequency component data is subjected toinverse two-dimensional Fourier transform processing, for example, animage S102 shown in FIG. 16E is obtained. For example, a pixel value ofthe image data S102 is subjected to binarization processing (forexample, rounding up 6 or more and rounding off 5 or less), image dataS103 shown in FIG. 16F is obtained.

FIG. 17A to FIG. 17E are views for explaining an operation of secondlow-pass filter processing of the low-pass filter portion.

The low-pass filter portion 1811 sets a region being larger than theregion ar_LPF1 as a threshold value of low-pass filter processing andperforms filter processing for a plurality of times.

The low-pass filter portion 1811 sets a region being larger than theregion ar_LPF1, for example, shown in FIG. 17A as explained above, suchas a region ar_LPF2, for example, shown in FIG. 17B.

In the second low-pass filter processing, specifically, as the thresholdvalue, for example as shown in FIG. 17B, a region ar_LPF2 surrounded by(180, 156), (156, 180), (−156, 180), (−180, 156), (−180, −156), (−156,−180), (156, −180), (180, −156) is set in the two-dimensional Fourierspace.

In the second low-pass filter processing, for example, as image dataafter the first low-pass filter processing, based on the image data S102shown in FIG. 16C and FIG. 17C, the low-pass filter portion 1811specifies low frequency component data in the region ar_LPF2 in theFourier space shown in FIG. 17B. For example, when the specified lowfrequency component data is subjected to two-dimensional Fouriertransform processing, an image S104 shown in FIG. 17D is obtained. Forexample, when a pixel value of the image data S104 is subjected tobinarization processing (for example, rounding up 6 or more and roundingoff 5 or less), image data S105 shown in FIG. 17E is obtained.

FIG. 18A to FIG. 18E are views for explaining an operation of a thirdlow-pass filter processing of the low-pass filter portion.

As third low-pass filter processing, the low-pass filter portion 1811sets a region being larger than the region ar_LPF2 shown in FIG. 18A asexplained above, a region ar_LPF3, for example as shown in FIG. 18B.

In the third low-pass filter processing, specifically, as the thresholdvalue, for example as shown in FIG. 18B, a region ar_LPF3 surrounded by(180, 157), (157, 180), (−157, 180), (−180, 157), (−180, −157), (−157,−180), (157, −180), (180, −157) is set in the two-dimensional Fourierspace.

In the third low-pass filter processing, for example, as image dataafter the second low-pass filter processing, based on the image dataS104 shown in FIG. 17D and FIG. 18A, the low-pass filter portion 1811specifies low frequency component data in the region ar_LPF3 in theFourier space shown in FIG. 18A.

For example, when the specified low frequency component data issubjected to inverse two-dimensional Fourier transform processing, animage S106 for example shown in FIG. 18D is obtained. For example, whena pixel value of the image data S106 is subjected to binarizationprocessing (for example, rounding up 6 or more and rounding off 5 orless), image data S107 shown in FIG. 18E is obtained.

FIG. 19A to FIG. 19F and FIG. 20A to FIG. 20C are views for explainingan operation of the low-pass filter portion of the data processingapparatus shown in FIG. 1. FIG. 21 is a flowchart for explaining anoperation of the low-pass filter portion of the data processingapparatus shown in FIG. 1. With reference to FIG. 14C, FIG. 19A to FIG.19F, FIG. 20A to FIG. 20C and FIG. 21, an operation of the low-passfilter portion 1811 will be explained.

In a step ST31, the low-pass filter portion 1811 performstwo-dimensional Fourier transform processing on the image data S1810,for example, shown in FIG. 14C and FIG. 19A as the first low-pass filterprocessing, sets a region ar_LPF1 to cut corners ar_h as high frequencycomponents in the Fourier space, specifies low frequency component datain the region ar_LPF1, performs inverse two-dimensional Fouriertransform processing, and generates image data S18011 shown in FIG. 19B(ST32). For example, when the image data S18011 is subjected tobinarization processing (for example, rounding up 6 or more and roundingoff 5 or less), image data S18103 shown in FIG. 19C is obtained.

In a step ST33, as the second low-pass filter processing, the low-passfilter portion 1811 performs two-dimensional Fourier transformprocessing based on the image data S18102 shown in FIG. 19B and FIG.19D, sets a region ar_LPF2, for example shown in FIG. 17B, for example,being larger than the region ar_LPF1, specifies low frequency componentdata in the region ar_LPF2, performs inverse two-dimensional Fouriertransform processing, and generates image data S18014 shown in FIG. 19E(ST33). For example, when binarization processing (for example, roundingup 6 or more and rounding off 5 or less) is performed on the image dataS18014, image data S18105 shown in FIG. 19F is obtained.

In a step ST34, as the third low-pass filter processing, based on theimage data S18104 shown in FIG. 19E and FIG. 20A, the low-pass filterportion 1811 performs two-dimensional Fourier transform processing,sets, for example, a region being larger than the region ar_LPF2, forexample a region ar_LPF3 shown in FIG. 18B (ST34), specifies lowfrequency component data in the region ar_LPF3 (ST35), performs inversetwo-dimensional Fourier transform processing to generate image dataS18106 shown in FIG. 20B, performs binarization processing (for example,rounding up 6 or more and rounding off 5 or less) on the image dataS18106, and generates the image data S1811 shown in FIG. 19F.

As explained above, as a result that the low-pass filter portion 1811specifies low frequency components comparing with the threshold value toleave a linear pattern by frequency components in the two-dimensionalFourier space obtained by performing two-dimensional. Fourier transformprocessing on image data so as to leave the linear pattern in the imagedata, and the specified low frequency component data is subjected toinverse two-dimensional Fourier transform processing; a linear patterncan be extracted. Also, by removing high frequency component datacomparing with the threshold value, it is possible to remove ageometrically symmetric pattern, for example, an approximately circularpattern.

Also, since the low-pass filter portion 1811 performs low-pass filterprocessing for a plurality of times by making the filter region ar_LPFlarger, a linear pattern can be extracted with higher accuracy.

FIG. 22A to FIG. 22C are views for explaining an operation of a maskportion and a skeleton portion of the data processing apparatus shown inFIG. 1.

The data processing apparatus 1 extracts a region to be used forauthentication from image data. In the present embodiment, the dataprocessing apparatus 1 extracts a region including a pattern indicatingblood vessels in image data as a region to be used for authentication.

The mask portion 1812 extracts a region P_N to be used forauthentication, for example, in the image data S1811 shown in FIG. 20C,and removes a pattern P_ct not to be used for authentication.

Specifically, based on the image data S1811, the mask portion 1812generates a mask pattern P_M as shown in FIG. 22A to extract the regionP_N to be used for authentication in the image data S1811, extracts aregion indicated by the mask pattern P_M from the image data S1811, andgenerates image data S1812, for example, shown in FIG. 22B.

The skeleton portion 1813 generates image data S1813 by performingskeleton processing based on the image data S1812. Also, the skeletonportion 1813 outputs the image data S1813 as a signal S102 to theauthentication unit 103.

Specifically, the skeleton portion 1813 performs degeneration processingby using the Morphology function based on the image data S1812, forexample, shown in FIG. 22B, makes a pattern indicating blood vesselsthin and generates image data S1813 obtained by extracting only thecenter portion of the pattern. The image data S1813 shown in FIG. 22Cindicates an image, wherein black and white are inversed for a plainexplanation.

Based on the signal S102 from the extraction unit 102, theauthentication unit 103 performs matching processing with registeredimage data D_P stored in the memory unit 17 in advance, and performsauthentication processing.

FIG. 23 is a flowchart for explaining an overall operation of the dataprocessing apparatus shown in FIG. 1. With reference to FIG. 23, anoperation of the data processing apparatus 1 will be explained plainly.In the present embodiment, an explanation will be made on the case wherea picture of a body of a subject h, for example a finger, is taken togenerate image data, a pattern indicating veins of the finger in theimage data is extracted, and authentication processing is performedbased on the pattern.

In a step ST101, the CPU 18 makes the irradiation portion 1011 of theimage pickup system 101 irradiate a near infrared ray to the finger ofthe subject h. In the image pickup unit 11, RGB image data S11 isgenerated based on a transmitted light inputted through the subject hand the optical lens 1012.

In a step ST102, the gray scale conversion portion 1801 performs, forexample, conversion to gray scale of 256 tones based on the RGB signalS11, and outputs this as a signal S1801 to the distribution datageneration portion 1802.

In the present embodiment, the image pickup system 101 generates the RGBimage data S11, but the present invention is not limited to thisembodiment. For example, in the case where the image pickup system 101generates gray scale image data S11, processing of the gray scaleconversion portion 1801 in the step ST102 is not performed and the imagedata S11 is output to the distribution data generation portion 1082.

In a step ST103, in the distribution data generation portion 1802, basedon the signal S1801, for example, when assuming the abscissa axis c is atone value (also referred to as a pixel value) and the ordinate axis fis the number (also referred to as a degree) of the pixel data, forexample as shown in FIG. 4C, histogram is generated as distribution datad1 for pixel data indicating pixel values in a 256-tone range as a firstrange r1.

In a step ST104, in the specifying portion 1803, based on the signalS1802, for example as shown in FIG. 5A, for the distribution data d1, arange of not more than the maximum pixel value of r11 among pixel valuesr11, r12, r13 and r14 by the number of predetermined threshold valueV_th in the first range r1 is specified as a second region r2, andoutputs this as a signal S1803.

Based on the signal S1803, the mapping portion 1804 maps pixel data inthe second region r2 specified by the specifying portion 1803 among aplurality of pixel data to the first region r1, generates second imagedata composed of the mapped pixel data, and outputs this as a signalS1804 to the Gaussian filter 1805.

Specifically, for example in the case where a range of pixel values of 0to 110 is the second range r2 as shown in FIG. 4C, the mapping portion1804 performs mapping by enlarging the pixel data to the first range r1as a range of pixel values of 0 to 256 as shown in FIG. 4D, enlarges thecenter portion of the image data including blood vessel information asshown in FIG. 4E, and generates second image data S1804 (ST105).

In a step ST106, for example, the selection portion 1814 detects noisedistribution characteristics of the signal S1804, and outputs to thenoise removing filter 1815 a signal S1814 to select any (at least one)noise removing filter suitable to the noise characteristics from aplurality of noise removing filters based on the detection results. Forexample, the selection portion 1814 outputs to the noise removing filter1815 a signal S1814 to select the Gaussian filter 1815_1 and theGaussian Laplacian filter 1815_10 for the noise removing processing.

In the noise removing filter 1815, any noise removing filter is selectedin accordance with the signal S1814 and, for example, the Gaussianfilter 1815_1 and the Gaussian Laplacian filter 1815_10 are selected.For convenience of the explanation, the Gaussian filter 1815_1 and theGaussian Laplacian filter 1815_10 are respectively explained as theGaussian filter 1805 and the Gaussian Laplacian filter 1806.

The Gaussian filter 1805 performs noise removing processing shown in theformulas (1) and (3) based on the signal S1804, for example, shown inFIG. 10A, generates image data S1805, for example, shown in FIG. 10B andoutputs to the Gaussian Laplacian filter 1806.

In a step ST107, the Gaussian Laplacian filter 1806 performs edgeenhancement processing based on the signal S1805, for example, shown inFIG. 10B, generates and outputs image data S1806, for example, shown inFIG. 10C. At this time, the image data S1806 is a binarized image data.

The Gaussian Laplacian filter 1806 performs binarization processingbased on a threshold value regulated in the first region r1 shown inFIG. 4C when performing binarization processing.

In a step ST108, based on the image data S1806, for example, shown inFIG. 10C, the first degeneration processing portion 1807 generates imageS1807 as shown in FIG. 14A by using as the focused pixel g_att theminimum pixel data among pixel data in a cross-shaped element EL1 as thefirst region, wherein the focused pixel data is at the center, forexample, as shown in FIG. 13A.

In a step ST109, based on the image data S1807, for example, shown inFIG. 14A, the first expansion processing portion 1808 generates imagedata S1808 as shown in FIG. 14B by using as the focused pixel g_att themaximum pixel data among pixel data in a cross-shaped element EL1 as thefirst region, wherein the focused pixel data is at the center, forexample, as shown in FIG. 13A.

In a step ST110, based on the image data S1808, for example, shown inFIG. 14B, the second expansion processing portion 1809 generates imagedata S1809 by using as the focused pixel g_att the maximum pixel dataamong pixel data in a 3×3 rectangular shaped element EL2 being largerthan the first region, for example, as shown in FIG. 13D, wherein thefocused pixel data is at the center.

In a step ST111, for example, based on the image data S1809, forexample, as shown in FIG. 13D, the second degeneration processingportion 1810 generates image data S1810, for example, as shown in. FIG.14C by using as the focused pixel g_att the minimum pixel data amongpixel data in a 3×3 rectangular shaped element EL2, wherein the focusedpixel data is at the center.

In a step ST112, as first low-pass filter processing, the low-passfilter portion 1811 performs two-dimensional Fourier transformprocessing on the image data S1810, for example, shown in FIG. 14C andFIG. 19A, sets a region ar_LPF1 to cut corners ar_h as high frequencycomponents in the Fourier space, for example, as shown in FIG. 16C,specifies low frequency component data in the region ar_LPF1, andperforms inverse two-dimensional Fourier transform processing togenerate image data S18011 shown in FIG, 19B.

As second low pass filter processing, the low-pass filter portion 1811performs two-dimensional Fourier transform processing based on the imagedata S18102 shown in FIG. 19B and FIG. 19D, sets, for example, a largerregion than the region ar_LPF1, for example, a region ar_LPF2 shown inFIG. 17B, specifies low frequency component data in the region ar_LPF2,and performs inverse two-dimensional Fourier transform processing togenerate image data S18014 shown in FIG. 19E.

As third low-pass filter processing, the low-pass filter portion 1811performs two-dimensional Fourier transform processing based on the imagedata S18104 shown in FIG. 19E and FIG. 20A, sets, for example, a largerregion than the region ar_LPF2, for example, a region ar_LPF3 shown inFIG. 18B, specifies low frequency component data in the region ar_LPF3,performs inverse two-dimensional Fourier transform processing togenerate image data S18016 shown in FIG. 20B, performs binarizationprocessing (for example, rounding up 6 or more and rounding off 5 orless) on the image data S18016 (ST113) and generates image data S1811shown in FIG. 19F.

In a step ST114, to extract a region P_N to be used for authenticationfrom the image data S1811 based on the image data S1811, the maskportion 1812 generates a mask pattern P_M as shown in FIG. 22A, extractsa region indicated by the mask pattern P_M from the image data S1811,and generates image data S1812, for example, shown in FIG. 22B.

In the step ST114, the skeleton portion 1813 performs degenerationprocessing by using the Morphology function based on the image dataS1812, for example, shown in FIG. 22B, makes a focused pattern, forexample, a pattern indicating blood vessels thin as shown in FIG. 22,generates image data S1813 obtained by extracting only the centerportion of the pattern, and outputs as a signal S102 to theauthentication unit 103.

In the authentication unit 103, matching processing, for example, withpre-registered image data D_P stored in the memory unit 17 is performedbased on the signal S102, and authentication processing is performed.

As explained above, the data processing apparatus 1 generatesdistribution data by the distribution data generation portion 1802, forexample, as shown in FIG. 5A and FIG. 5B, specifies a second region bythe specifying portion 1803, maps the second region to the first regionby the mapping portion 1804, and generates third image data bybinarization based on a threshold value regulated in the first region r1by components 1805 to 1812; therefore, binarization can be suitablyperformed even in the case where distribution data d1 of pixel valuesdiffers for each subject h.

Also, as a result that the selection portion 1814 for selecting anynoise removing processing among a plurality of noise removing processingand a noise removing filter 1815, for example, having a plurality ofdifferent kinds of noise removing filters are provided, for example, afilter selected, for example, by the selection portion 1814 performsnoise removing processing based on the signal S1804, then, the GaussianLaplacian filter 1806 performs edge enhancement processing andbinarization; it is possible to remove noises caused by diffusedreflection of a body of the subject h and image pickup unit 11 and otherdevices, for example, from the image data S1804, and suitably binarizedimage data can be generated based on the predetermined threshold valueof the first region r1.

Also, for each of a plurality of pixel data composing the image dataS1806 and indicating pixel values, the first degeneration processingportion 1807 for generating image data S1807 by using as predeterminedpixel data the minimum pixel data among pixel data in the first regionaround the pixel data, the first expansion processing portion 1808 forgenerating image data S1808 by using as predetermined pixel data themaximum pixel data among pixel data in the first region around the pixeldata, the second expansion processing portion 1809 for generating imagedata S1809 by using as predetermined pixel data the maximum pixel dataamong pixel data in the second region being larger than the first regionaround the pixel data, and the second degeneration processing portion1810 for generating image data S1810 by using as predetermined pixeldata the minimum pixel data among pixel data in the second region beinglarger than the first region around the pixel data; it is possible toremove regions being smaller than a predetermined size and to connectbetween pixel data close to a certain extent to each other. Furthermore,it is possible to leave a linear pattern, and a pattern as noisecomponents can be removed.

Also, as a result that the low-pass filter portion 1811 specifies lowfrequency component data comparing with the threshold value to leave alinear pattern by frequency components in the two-dimensional Fourierspace obtained by performing two-dimensional Fourier transformprocessing on image data so as to leave the linear pattern in the imagedata, and performs inverse two-dimensional Fourier transform processingon the specified low frequency component data, it is possible to extracta linear pattern. Also, a geometrically symmetric pattern can beremoved.

Also, as a result of performing a series of processing operations, forexample, a pattern indicating blood vessels of a subject h can beextracted with high accuracy.

Also, it is possible to extract a pattern indicating individually uniquevein of blood vessels with high accuracy, so that authentication withhigher accuracy can be performed based on the pattern.

Also in a conventional data processing apparatus, troublesome processingof using an AI filter for blood vessel tracing based on blood vesselinformation from image data was performed. However, in the dataprocessing apparatus 1 according to the present embodiment, it ispossible to extract a pattern indicating blood vessels with highaccuracy based on image data obtained by taking a picture of a subjecth, so that a load on processing becomes lighter comparing with that inthe conventional case.

Also, the skeleton portion 1813 extracts the center portion of a patternindicating blood vessels when performing skeleton processing, thus, itis possible to generate skeleton image data less affected by expansionand contraction of blood vessel, for example, along with changes ofphysical conditions of the subject h. Since the authentication unit 103uses the image data for authentication processing, it is possible toperform authentication processing with high accuracy even when aphysical conditions of the subject h change.

Also, the present embodiment can be realized by combining filtersrequiring light processing, therefore, an individual authenticationsystem with a high processing speed can be developed.

FIG. 24 is a view for explaining a second embodiment of a remote-controldevice using the data processing apparatus according to the presentinvention.

A remote-control device (also referred to as a remote controller) 1 aaccording to the present embodiment is a general remote controllerprovided therein with the data processing apparatus 1 according to thefirst embodiment.

Specifically, as same as the data processing apparatus according to thefirst embodiment, for example, shown in FIG. 2, the remote-controldevice 1 a comprises an image pickup unit 11, an input unit 12, anoutput unit 13, a communication interface 14, a RAM 15, a ROM 16, amemory unit 17 and a CPU 18. An explanation will be made only ondifferent points from the data processing apparatus 1 according to thefirst embodiment.

In the remote-control apparatus 1 a, for example, an irradiation portion1011, an optical lens 1012 and the image pickup unit 11 as an imagepickup system 101 are provided to a body portion 100.

The output portion 13 transmits a control signal to make a televisionset m_tv perform predetermined processing by using an infrared ray as acarrier wave, for example, by being controlled by the CPU 18. Forexample, the output unit 13 is composed of an infrared ray lightemitting element.

The television set m_tv performs predetermined processing in accordancewith a control signal received at a light receiving portion m_r, forexample, displaying predetermined image on a display portion m_m.

The memory unit 17 stores, for example, data D_t indicating users'preferences, specifically, a preference list D_t as shown in FIG. 24.The data D_t is read and written in accordance with need by the CPU 18.

The CPU 18 performs processing, for example, in accordance with the dataD_t when authentication is performed normally.

FIG. 25 is a flowchart for explaining an operation of the remote-controldevice 1 a shown in FIG. 24.

In a step ST201, whether a user touched the image pickup system 101provided on the side surface of the body portion 100 is determined. Forexample, when a finger touched the image pickup system 101, theprocedure proceeds to processing in a step ST202.

In the step ST202, by the CPU 18, a near infrared ray is irradiated fromthe irradiation portion 1011 to a finger of the subject h, and the imagepickup unit 11 generates image data of finger blood vessels based on atransmitted light. In the present embodiment, a light irradiated fromthe irradiation portion 1011 is used, but the present invention is notlimited to this embodiment. For example, the image pickup unit 11 maygenerate image data based on the transmitted light of the subject h by anatural light.

In a step ST203, the CPU 18 extracts image data to be used forauthentication by the extraction unit 102, for example, skeleton imagedata indicating a pattern indicating blood vessels, and outputs as asignal S102 to the authentication unit 103.

In a step ST204, the CPU 18 makes the authentication unit 103 performauthentication processing by comparing the signal S102 with users'registered image data D_P stored in the memory unit 17 in advance.

In a step ST205, when the authentication unit 103 recognizes it is notthe user stored in advance, the procedure returns back to the processingin the step ST201.

On the other hand, when the authentication unit 103 recognizes it is theuser stored in advance in the determination in the step ST205, the CPU18 performs processing in accordance with the data D_t indicatingpreference of the user stored in the memory unit 17. For example, acontrol signal in accordance with the data D_t is output to thetelevision set m_tv.

As explained above, in the present embodiment, since a remote-controldevice comprising the data processing apparatus according to the firstembodiment is provided, for example, it is possible to control thetelevision set m_tv based on the authentication result.

Also, for example, age and other information are included in the dataD_t. When the user is determined to be underage as a result of theauthentication by the authentication unit 103, an age limiting functioncan be realized by performing limiting processing, such that the CPU 18invalidates a specific button to keep programs on the televisionblocked, etc.

Also, the data D_t includes display of program listing (a preferencelist and history, etc.) customized for each user and use of programmedrecording list, etc. The CPU 18 is capable of performing processing inaccordance with the respective users by controlling that the data can beused when authentication by the authentication unit 103 is normal.

Also, a plurality of predetermined data may be registered for each userin the data D_t.

FIG. 26 is a view for explaining a third embodiment of the dataprocessing system using the data processing apparatus according to thepresent invention.

A data processing system 10 b according to the present embodimentcomprises, as shown in FIG. 26, a remote-control device 1 a, a recordingmedium (also referred to as medium) 1 b, a data processing apparatus 1 cand a television set m_tv. An explanation will be made only on differentpoints from the first embodiment and the second embodiment.

In the present embodiment, for example, both of the remote-controldevice 1 a and the recording medium 1 b perform the above identificationprocessing, and processing in accordance with both of the identifiedresults is performed. For example, when a user of the remote-controldevice 1 a and a user of the recording medium 1 b are identical, readingand writing of predetermined data stored in the recording medium 1 b areperformed.

The remote-control device 1 a has the approximately same configurationas that of the remote-control device 1 a according to the secondembodiment and includes the data processing apparatus 1 according to thefirst embodiment.

The recording medium 1 b comprises, for example, the data processingapparatus 1 according to the first embodiment.

For example, the recording medium 1 b is a video tape and other magneticrecording media, an optical disk, a magneto-optical disk, asemiconductor memory, and other data recording media.

As same as in the first embodiment, for example shown in FIG. 2, therecording medium 1 b comprises an image pickup unit 11, an input unit12, an output unit 13, a communication interface 14, a RAM 15, a ROM 16,a memory unit 17 and a CPU 18. An explanation will be made only ondifferent points from the data processing apparatus 1 according to thefirst embodiment.

In the recording medium 1 b, an irradiation portion 1011, an opticallens 1012 and the image pickup unit 11 are provided as an image pickupsystem 101 to a body portion 100.

The image pickup system 101 is preferably provided at a position touchedby a user on the body portion 100 b. When a position touched by a useris not determined, the image pickup unit 11 is provided not only at oneposition but at regions that could possibly be touched by a user on thebody portion 100 b.

The data processing apparatus 1 c is capable of reading and writing datastored by the recording medium 1 b, for example, when authentication isperformed normally. For example, the data processing apparatus 1 cincludes the data processing apparatus according to the firstembodiment. For example, the data processing apparatus 1 c comprises, assame as in the first embodiment shown in FIG. 2, an image pickup unit11, an input unit 12, an output unit 13, a communication interface 14, aRAM 15, a ROM 16, a memory unit 17 and a CPU 18. An explanation will bemade only on different points from the data processing apparatus 1according to the first embodiment.

Furthermore, the data processing apparatus 1 c comprises, for example, aholding portion m_h for holding the recording medium 1 b, a driver forperforming reading and writing of data of the recording medium 1 b heldby the holding portion m_h, and a light receiving portion m_r, etc.

A television set m_tv comprises a display portion m_m for displaying animage based on data, for example, from the driver of the data processingapparatus 1 c.

FIG. 27 is a flowchart for explaining an operation of the dataprocessing system shown in FIG. 26. With reference to FIG. 27, anexplanation will be made only on different points from the firstembodiment and the second embodiment on an operation of the dataprocessing system 10 b.

An operation of the remote-control device 1 a in the steps ST301 toST304 is the same as that in the steps ST201 to ST204 in the secondembodiment, so that the explanation will be omitted.

In a step ST304, the CPU 18 of the remote-control device 1 a makes theauthentication unit 103 compare a signal S102 with registered image dataD_P of a plurality of users stored in advance in the memory unit 17 toperform authentication processing.

In a step ST305, when it is identified to be not a user stored inadvance in the authentication unit 103 of the remote-control device 1 a,the procedure returns back to the processing in the step ST301.

On the other hand, in determination in the step ST305, when it isidentified to be a user stored in advance in the authentication unit103, the CPU 18 stores, for example, the identified result is stored asA in the memory unit 17 (ST306).

In the step ST307, for example, a user sets the recording medium 1 b atthe holding portion m_h of the data processing apparatus (also referredto as a reproduction apparatus) 1 c.

In a step ST308, in the recording medium 1 b, for example, whether theuser touched the image pickup system 101 provided on the side surface ofthe body portion 100 b is distinguished. When, for example, a fingertouched the image pickup system 101, the procedure proceeds toprocessing in a step ST309.

In the step ST309, the CPU 18 of the recording medium 1 b makes theirradiation portion 1011 irradiate a near infrared ray to the finger ofthe subject h and makes the image pickup unit 11 generate image data ofthe finger vein based on a transmitted light.

In a step ST310, the CPU 18 of the recording medium 1 b extracts by theextraction unit 102 image data, for example, skeleton image dataindicating a pattern indicating blood vessels to be used forauthentication in the same way as in the first embodiment, and outputsthis as a signal S102 to the authentication unit 103.

In a step ST 311, the CPU 18 of the recording medium 1 b makes theauthentication unit 103 compare the signal S102 with the registeredimage data D_P of a plurality of users stored in advance in the memoryunit 17 to perform authentication processing.

In a step ST312, when it is identified to be not a user stored inadvance in the authentication unit 103 of the recording medium 1 b, theprocedure returns back to the processing of the step ST308.

On the other hand, in determination in the step ST312, when it isidentified to be a user stored in advance in the authentication unit 103of the recording medium 1 b, the CPU 18 of the recording medium 1 b setsthe identified result as B (ST313).

In a step ST314, the identified result A by the step ST306 and theidentified result B by the step ST313 are compared and determinedwhether they are an identical user or not.

The determination processing may be performed in the recording medium 1b. In this case, the recording medium 1 b performs processing based onthe identified result A sent from the remote-control device 1 a and theidentified result B by the recording medium 1 b.

Also, the determination processing may be performed, for example, in thedata processing apparatus 1 c. In this case, the data processingapparatus 1 c performs processing based on the identified result A sentfrom the remote-control device 1 a and the identified result B by therecording medium 1 b.

When determined to be an identical user as a result of the determinationprocessing in the step ST314, for example, the recording medium 1 ballows the data processing apparatus 1 c to read and write, for example,to reproduce and record built-in data (ST315), while when determined tobe not an identical user, for example, the recording medium 1 bprohibits the data processing apparatus 1 c to reproduce and record thebuilt-in data (ST316).

For example, when determined to be an identical user, for example, thedata processing apparatus 1 c reads data built in the recording medium 1b and makes the television set m_tv display an image in accordance withthe data on the display portion m_m.

As explained above, in the present embodiment, since identification isperformed by both of the remote-control device 1 a and the recordingmedium 1 b, for example, when it is an identical user from theidentified result, it is possible to make the recording medium 1 b storeand read data. Therefore, for example, data falsifying, a stealthyglance and overwriting on data, etc. by other people can be prevented.

FIG. 28 is a view for explaining a fourth embodiment of a portablecommunication device using the data processing apparatus according tothe present invention.

The portable communication device 1 d according to the presentembodiment includes the data processing apparatus according to the firstembodiment.

For example, the portable communication device 1 d has a general callfunction, an email function and an address book function, etc. andactivates a predetermined function in the case of a pre-registered useras a result of the above authentication processing, but does notactivate a predetermined function in the case of a not registered user.

The portable communication device 1 d comprises, for example, as same asin the first embodiment shown in FIG. 2, an image pickup unit 11, aninput unit 12, an output unit 13, a communication interface 14, a RAM15, a ROM 16, a memory unit 17 and a CPU 18. An explanation will be madeonly on different points from the data processing apparatus 1 accordingto the first embodiment.

In the portable communication device id, an image pickup system 101 isprovided, for example, to a call button bt, etc. (may be all buttons bt)as the input unit 12.

For example, the portable communication device 1 d obtains an image offinger vein when a button bt is operated by the user and, when it isidentified to be an already registered individual, activates acommunication function as a cellular phone, and activates a desired callfunction via a not shown base station.

FIG. 29 is a flowchart for explaining an operation of the dataprocessing apparatus shown in FIG. 28. With reference to FIG. 29, anoperation of the portable communication device 1 d will be explained ondifferent points from the data processing apparatus according to thefirst to third embodiments.

In a step ST401, for example, whether a user touched the image pickupsystem 101 provided to a call button bt, etc. as the input unit 12 ornot is distinguished. When, for example, a finger touched the imagepickup system 101, the procedure proceeds to a step ST402.

In a step ST402, the CPU 18 makes the irradiation portion 1011 irradiatea near infrared ray to the finger of the subject h and makes the imagepickup unit 11 generate image data of the finger vein based on atransmitted light.

In a step ST403, the CPU 18 extracts by the extraction unit 102 imagedata, for example, skeleton image data indicating a pattern indicatingblood vessels to be used for authentication in the same way as in thefirst embodiment, and outputs this as a signal S102 to theauthentication unit 103.

In a step ST 404, the CPU 18 makes the authentication unit 103 comparethe signal S102 with the registered image data D_P of users stored inadvance in the memory unit 17 to perform authentication processing.

In a step ST405, when it is identified to be a pre-registered user inthe authentication unit 103, a communication function as a cellularphone is activated and the user is permitted to use the cellular phone(ST406).

For example, when a user as a holder of the portable communicationdevice 1 d lends the portable communication device 1 d to other person(ST407), the portable communication device 1 d determines whether theuser as a holder operated a specific button (ST408).

In determination in the step ST408, when it is determined that thespecific button bt is operated, the CPU 18 makes the predeterminedfunction able to be operated by other person. The user as a holder lendsthe portable communication device 1 d in that state to other person(ST409).

On the other hand, in the step ST408, in the case, where the CPU 18determines that the specific button bt is not operated by the identicalperson, and in the case of not lending to other person, a series ofprocessing finishes.

On the other hand, in the determination in the step ST405, when it isidentified to be not a pre-registered user in the authentication unit103 and when the identification processing is not failed for a pluralityof times (ST410), the procedure returns back to the step ST401.

On the other hand, when the identification processing failed for aplurality of times, the CPU 18 prohibits the authentication processing(ST411), and the CPU 18 sends data indicating that the authenticationprocessing failed for a plurality of times, for example, to a datacommunication device PC registered in advance (ST412).

As explained above, in the present embodiment, the portablecommunication device 1 d activates a predetermined function when it isapproved to be a pre-registered user as a holder from the result of theabove authentication processing, it is possible to prevent the devicefrom being used by other people, for example, in case of loss.

Also, for example, in the case of not a pre-registered user, receivedemails, not to mention the address book, and transmission history, etc.cannot be viewed, so that the security is tight.

Also, by registering data indicating an address, such as a mail address,in advance in the memory unit 17 and providing, for example, a GPS(global positioning system) function, information of a present positionof the portable communication device 1 d can be sent to a datacommunication device PC at an already registered address, for example,when keys are pressed for a plurality of times by someone other than thepre-registered user (when authentication failed for a plurality oftimes).

Also, by storing address book data ad of the user in a server device svaccessible via a not shown communication network separately from theportable communication device 1 d and, for example, when the user isappropriately authenticated in the portable communication device 1 d,accessing by the CPU 18 of the portable communication apparatus 1 d tothe server device sv via the not shown communication network to downloadthe user's address book data, it is possible to prevent other users fromimproperly browsing the address book.

In this case, for example, even when other portable communication device1 d is used, it is possible to use the same address book data ad whenauthentication is appropriately attained by the portable communicationdevice 1 d.

Also, when the user as a holder of the portable communication device 1 dneeds to lend the portable communication device 1 d to other person withown acknowledgement, it is possible to allow other person to use it byan operation of an exclusive button bt by the holder himself/herself.Namely, when a specific button bt is operated, the CPU 18 activates apredetermined function without performing authentication processing evenin the case of being used by other person.

FIG. 30 is a view for explaining a fifth embodiment of the dataprocessing apparatus according to the present invention.

A telephone 1 e using the data processing apparatus according to thepresent embodiment comprises the data processing apparatus 1 accordingto the first embodiment and has an individual authentication function byfinger vein.

For example, in the telephone 1 e according to the present embodiment,an image pickup system 101 is provided to a specific button bt, etc.(may be all buttons or its body) provided to the respective household inthe same way as in the portable communication device 1 d according tothe fourth embodiment. The configuration of the telephone 1 e isapproximately the same as that of the portable communication device 1 daccording to the fourth embodiment. Only different points will beexplained.

The telephone 1 e obtains an image of finger vein, for example, when abutton bt is pressed. Also, the telephone 1 e has a use limitingfunction and an individual identification function.

Also, the telephone 1 e has a limiting function for setting the maximumusable time for each user in advance and inactivating calls whenreaching to a predetermined time.

Also, for example in the case of a length phone call, when it is set toperform authentication only when a button is pressed for the first time,it is not always the case that the same person would use afterwards, sothat it is preferable to provide the image pickup system 101, forexample, to a receiver 1 e_r and take a picture of finger vein of asubject h regularly to continuously perform authentication processing.

Also, the telephone 1 e regularly performs authentication processing andupdates registered image data in the memory unit 17.

Also, for example, in the telephone 1 e, different beep sound can beregistered for each user, and, for example, a beep sound correspondingto the user can be output from a not shown speaker.

FIG. 31 is a flowchart for explaining an operation of the telephoneshown in FIG. 30. With reference to FIG. 31, an operation of thetelephone 1 e will be explained. For example, the case where thetelephone 1 e is used by a plurality of users will be explained.

In a step ST501, when the CPU 18 receives a signal indicating callingfrom other telephone, it specifies the user based on the other party'stelephone number, etc. and let beep sound corresponding to the useroutput from the speaker.

Whether individual identification by beep sound is used or not isdetermined (ST502) and, when individual identification is used, the CPU18 makes the image pickup system 101 irradiate a light to a user'sfinger and take a picture of the finger (ST504) when the finger of theuser corresponding to the beep sound touches the image pickup system 101of a receiver, etc. (ST503).

In a step ST505, the same processing as in the data processing apparatus1 according to the first embodiment is performed and image dataincluding a pattern indicating blood vessels is generated.

In a step ST506, the CPU 18 compares the generated image data with alist of registered image data stored in the memory unit 17.

In a step ST507, identification timeout or not, that is, whether aprocessing time taken for identification processing is longer than apredetermined time or not, is determined; and whether identification isattained or not is determined if it is within the processing time(ST508).

When identification is appropriately attained by the determination inthe step ST507, the CPU 18 sets to enable to call (ST509).

On the other hand, when identification cannot be attained appropriatelyby the determination in the step ST507, the procedure returns back tothe step ST504 to repeat measurement. But when it becomes timeout in thestep ST507, it is switched, for example, to a so-called answerphonefunction (ST510).

On the other hand, in the step ST502, it is also switched to ananswerphone function in the same way when the beep sound is not his/herown.

On the other hand, in the step ST501, in the case of not receiving acall but making a call, the CPU 18 determines whether, for example, auser's finger touched the image pickup system 101 (ST551). When, forexample, a finger touched the image pickup system 101, the procedureproceeds to a step ST512.

In the step ST512, the CPU 18 makes the irradiation portion 1011irradiate a near infrared ray to the finger of the subject h, and makesthe image pickup unit 11 generate image data of finger vein based on atransmitted light.

In a step ST513, the CPU 18 extracts by the extraction unit 102 imagedata to be used for authentication, for example, skeleton image dataindicating a pattern indicating blood vessels in the same way as in thefirst embodiment, and outputs as a signal S102 to the authenticationunit 103.

In a step ST514, the CPU 18 makes the authentication unit 103 comparethe signal S102 with the registered image data D_P of users stored inadvance in the memory unit 17 to perform authentication processing.

In a step ST515, when it is identified to be not an already stored userin the authentication unit 103, the procedure returns back to theprocessing in the step ST511.

On the other hand, when it is identified to be an already stored user inthe authentication unit 103 in the determination in the step ST515, theCPU 18 displays on the display portion of the output unit 13 addressbook data ad of the identified user (ST516) and sets to enable to call(ST517).

For example, during a call, in the step S518, whether a use time is setor not is determined.

When it is set, the CPU 18 determines whether it is within a usable timeor not (ST519) and determines whether the call is ended when it iswithin the usable time (ST520).

On the other hand, in the step ST519, when it is not within the usabletime, an alert or indication of an alert is displayed on the displayportion for the user, and the call is forcibly disconnected (ST521).

On the other hand, in the step ST518, when the use time is not set andwhen the call ended in the step ST520, a series of processing finishes.

As explained above, in the present embodiment, the telephone 1 e has thedata processing apparatus according to the first embodiment therein, anda use time can be set, so that, for example, a lengthy phone call can beprevented.

Also, it is possible to apply a system wherein billing of phone chargesis divided for the respective users and the payment of the phone chargesis made by each user.

Also, the image pickup system 101 was provided to a button bt, but thepresent invention is not limited to this embodiment. For example, it maybe provided to a receiver 1 e_r, etc. or provided to both of the buttonbt and the receiver 1 e_r, and the both may be used appropriatelydepending on the situation.

Also, being different from the portable communication device, thetelephone 1 e can be used by a plurality of users, for example, allfamily members can use the same telephone, and it displays an addressbook for an identified user, preferable operationality is attained.

Also, when receiving a call, since beep sound can be set for each userand it can be set that only an identified individual can answer thephone, the security is tight.

Also, by performing individual identification at the time of picking upthe receiver, and setting to enable to talk in the case of a presetuser, tight security is attained.

Also, when the preset user is absence, it can be switched to ananswerphone, for example, even if other family is at home, so that tightsecurity is attained.

FIG. 32 is a view for explaining a sixth embodiment of the dataprocessing apparatus according to the present invention.

A PDA (personal digital assistant) 1 f according to the presentembodiment comprises the data processing apparatus 1 according to thefirst embodiment.

For example, as shown in FIG. 32, in the PDA 1 f, an image pickup system101 is provided to a side surface of the body portion or a button bt.

For example, when a user touches the PDA 1 f, an image of finger vein isobtained, and it can be used in the case of an identical person as aresult of authentication.

Alternately, it is set that private data can be displayed only in thecase of an identical person as a result of authentication.

FIG. 33 is a view for explaining a seventh embodiment of the dataprocessing apparatus according to the present invention.

A mouse 1 g according to the present embodiment is a so-called mouse asan input device of, for example, a personal computer PC, etc. andcomprises the data processing apparatus 1 according to the firstembodiment.

In the mouse 1 g, an image pickup system 101 is provided, for example,to a button bt, etc.

For example, when a user touches a button bt of the mouse 1 g, an imageof finger vein is obtained, and it is set that login to the personalcomputer is possible only in the case of an identical person as a resultof authentication. For example, it may be used for turning on the powerof the personal computer PC and displaying a login screen, etc.

Note that the present invention is not limited to the present embodimentand a variety of any suitable modifications can be made.

An explanation was made on an example wherein the data processingapparatus 1 was built in a remote-control device and a portablecommunication device, but the present invention is not limited to theseembodiments.

For example, a keyboard may be provided with the image pickup system 101to take a picture of a subject h during key input, and authenticationprocessing may be performed based on the taken image data.

Also, when using net-shopping, etc., a picture of a subject h is takenduring inputting necessary information, and authentication processingmay be performed based on the taken image data. In this case, it is madeto be a mechanism wherein only an identical person can make an order.Also, by using it together with the credit card number and password,etc., double management can be attained and the security is furthermoreimproved.

Also, the image pickup system 101 may be provided to a touch panel of,for example, a bank ATM (automatic teller machine), etc. A picture of asubject h is taken when inputting necessary information, andauthentication processing may be performed based on the taken imagedata. For example, by setting that cash can be withdrawn when a personis identified to be an identical person, the security is improved.

Also, when using together with a cash card and password, etc., thesecurity is furthermore improved.

Also, the image pickup system 101 may be provided, for example, to ahouse key and a mail post, etc. to take a picture of a subject h, andauthentication processing may be performed based on the taken imagedata. By providing a mechanism of opening a door when appropriatelyidentified, the security is improved. Also, when it is used togetherwith a key, etc., the security is furthermore improved.

Also, the data processing apparatus 1 may be provided, for example, to abicycle to take a picture of a subject h, and authentication processingmay be performed based on the taken image data. By providing a mechanismof turning on and off of the key when appropriately identified, thesecurity is improved. Also, when it is used together with the key, thesecurity is furthermore improved.

Also, for example, it may be used in place of signature when using acredit card. For example, by providing the data processing apparatus 1to a reader/writer of a credit card, etc., it becomes possible todisplay the card number, for example, when identification isappropriately attained, to confirm matching with own card.

Also, by using together with a card, etc., the security is furthermoreimproved. Also, by using them together, it is possible to preventmisusage of the card or key in case of loss.

INDUSTRIAL APPLICABILITY

The present invention can be applied to, for example, an imageprocessing apparatus for performing processing on image data obtained bytaking a picture of a subject.

1. An image processing apparatus, comprising: a distribution datageneration means for generating distribution data indicating adistribution of pixel data for a plurality of pixel data indicatingpre-regulated pixel values in a first range and composing first imagedata obtained by taking a picture of a subject; a specifying means forspecifying a second range to be binarized in the first range based onthe distribution data generated by the distribution data generationmeans; a mapping means for mapping pixel data in the second rangespecified by the specifying means among a plurality of pixel data to thefirst range, and generates second image data composed of the mappedpixel data; and a binarization means for binarizing the second imagedata generated by the mapping means based on a threshold value regulatedin the first range to generate a third image data.
 2. An imageprocessing apparatus as set forth in claim 1, wherein said distributiondata generation means generates a distribution data indicating thenumber of pixel data having pixel values for the respective pixel valuesin the first range; and said specifying means specifies as the secondrange a range having pixel values which is maximum pixel value or lessamong predetermined number of the pixel data within the first range. 3.An image processing apparatus as set forth in claim 1, furthercomprising a noise removing filter means for generating a fourth imagedata by performing edge enhancement processing after noise removingprocessing on the second image data generated by the mapping means,wherein the binarization means binaries the fourth image data generatedby the noise removing filter means based on a threshold value regulatedin the first range to generate the third image data.
 4. An imageprocessing apparatus as set forth in claim 3, wherein said noiseremoving filter means generates the fourth image data by performingGaussian Laplacian filter for the edge enhancement processing, afterperforming Gaussian filter for the noise removing processing on thesecond image data generated by the mapping means.
 5. An image processingapparatus as set forth in claim 4, wherein said noise removing filtermeans generates the fourth image data by performing any one processingamong Gaussian filter, maximum value filter, minimum value filter,two-dimensional adaptive noise removing filter, proximity filter,averaging filter, Gaussian low-pass filter, two-dimensional Laplacianproximity filter, or Gaussian Laplacian filter for noise removingprocessing based on the second image data generated by the mappingmeans.
 6. An image processing apparatus as set forth in claim 1, furthercomprising a filter processing means for generating a fifth image databy performing filter processing to leave a linear pattern based on thesecond image data generated by the mapping means, wherein thebinarization means binaries the fifth image data generated by the filterprocessing means based on a threshold value regulated in the first rangeto generate the third image data.
 7. An image processing apparatus asset forth in claim 6, wherein said filter processing means specifies thefifth image data of low frequency components compared with the thresholdvalue to leave a linear pattern by frequency components in thetwo-dimensional Fourier space obtained by performing two-dimensionalFourier transform processing on the second image data generated by themapping means.
 8. An image processing method, including: a first stepfor generating distribution data indicating a distribution of pixel datafor a plurality of pixel data indicating pre-regulated pixel values in afirst range and composing first range data obtained by taking a pictureof a subject; a second step for specifying a second range to bebinarized in the first range based on the distribution data generated bythe first step; a third step for mapping data in the second rangespecified by the second step among a plurality of pixel data to thefirst range, and generates second image data composed of the mappedpixel data; and a fourth step for binarizing the second image datagenerated by the third step based on a threshold value regulated in thefirst range to generate a third image data.
 9. An image processingmethod as set forth in claim 8, wherein said first step generates thedistribution data indicating the number of pixel data having pixelvalues for respective pixel values in the first range; and said secondstep specifies as the second range a range having pixel data which ismaximum pixel value or less among a predetermined number of the pixeldata within the first range as the second range to be binarized.
 10. Animage processing apparatus as set forth in claim 9, further comprising afifth step of generating fourth image data by performing edgeenhancement processing after performing noise removing processing basedon the second image data generated by the third step, wherein the fourthstep generated by the fifth step on a threshold value regulated in thefirst range to generate the fourth image data.
 11. An image processingmethod as set forth in claim 10, wherein said fifth step generates thefourth image data by performing Gaussian Laplacian filter for the edgeenhancement processing after performing Gaussian filter for the noiseremoving processing on the second image data generated by the thirdstep.
 12. An image processing method as set forth in claim 10, whereinsaid fifth step generates the fourth image data by performing anyprocessing among a plurality of different noise removing processingbased on the second image data generated by the third step.
 13. An imageprocessing method as set forth in claim 12, wherein saaid fifth stepgenerates the fourth image data by performing any one processing amongGaussian filter, maximum value filter, minimum value filter,two-dimensional adaptive noise removing filter, proximity filter,averaging filter, Gaussian low-pass filter, two-dimensional Laplacianproximity filter, or Gaussian Laplacian filter for noise removingprocessing on the second image data generated by the third step.
 14. Animage processing method as set forth in claim 8, further comprising asixth step for generates the fifth image data by performing filterprocessing to leave a linear pattern based on the second image datagenerated by the third step, wherein the fourth step binaries the fifthimage data generated by the sixth step based on a threshold valueregulated in the first rage to generate the third image data.
 15. Animage processing method as set forth in claim 14, wherein said sixthstep specifies the fifth image data of low frequency components comparedwith the threshold value to leave a linear pattern by frequencycomponents in the two-dimensional Fourier transform processing on thesecond image data generated by the third step, and the fourth stepbinaries the fifth image data specified by the sixth step on a thresholdvalue regulated in the first range to generate the third image.
 16. Animage processing method as forth in claim 15, wherein said sixth stepspecifies the fifth image data of low frequency components comparingwith the threshold value larger than the threshold value by frequencycomponents in the two-dimensional Fourier space obtained by performingtwo-dimensional Fourier transform processing on the image data generatedby the third step.